Jump to content

User Notes -- Corrupted


William Phillips

Recommended Posts

Well I am sickened to write this but my user notes -- such as they were -- seem absolutely corrupted beyond any retrieval. Unfortunately the model used by concordance for these files (seemingly no proper way to export files, a confusing import and merge process that seems to just further corrupt what hope one had of retrieving files, a reliance on dropbox which is hopeless, and so forth) have resulted in utter confusion and apparently forever lost notes.

 

Although I suppose Technical Support could possibly still help I am appalled that such an important feature should be obscured into some opaque database that is coded in such a way as to make looking within impossible. And now I find when I even tried to merge user notes (in the vain hope that would accomplish something) it winds up deleting or perhaps just moving to the 'old' folder, the original -- but even those files display nothing.

 

Hopeless, confusing, infuriating. This methodology has now, in effect, stolen my own information!  I am very disappointed. I guess I may have to see  if any alternatives out there have a better, more understandable, standardized (as in file system manipulation) model. Even going to something like Logos (if any better) will not retrieve what I seem to have lost.

  • Sad 1
Link to comment
Share on other sites

William, sorry to hear this. By chance do you have a backup? Or were you perhaps syncing via Dropbox that might have left an older version in your DB folder?

 

Link to comment
Share on other sites

I have tried backups that were on my drive but they come up blank and then change the file name (add .acc4 to the name but that is about it.  Frustratingly I can't even seem to find a 'smooth' pathway to do this. I see a 'add folder' tool but that doesn't let me add a user file while the merge note tool is a disaster.  This is really not good. I've used many software programmes over the year of all types but I have never run into such roadblocks before. I don't know what the programmers or management were thinking but this is a disaster. There should be simple tools to do these things. I just don't get it.

  • Sad 1
Link to comment
Share on other sites

I finally figured out a way to get them back.  But it was not through the programme itself. I found what must be the last good copy I had somehow and used the file system to delete all the corrupted versions.  

 

So, it works again. But this design is not in the user's interest and I urge Accordance to change to a transparent method. After all, they are the users notes.

 

Thx..

  • Like 3
Link to comment
Share on other sites

  • 4 months later...

I have just lost all my user notes, and no version of backups seems to work. They all either come up blank, add the acc4 extension, and/or have I/O errors.  When I say 'lost', they are there, but they won't work. They are not empty, I can open them with a code editor and content is there, they simply no longer work with my Accordance. If they do load at all, they are empty in the software though they are not empty in the file.

 

This all started because my notes were  not syncing with iPad via dropbox. So I followed the steps advised by Accordance Tech support, and now I seem to have lost all my notes that I have collected over years.

 

Extremely frustrating. Accordance is a great product, and tech support is typically very helpful. But this case is extremely frustrating.

Edited by jkdoyle
clarify
  • Sad 1
Link to comment
Share on other sites

Sorry to hear of the trouble with Notes.  I have had similar issues and due to Technical Support not being able to help me recover, I've automated backups and manually copy Text to/from Scrivener, which allowed me to rebuild them.

 

I agree with William's assessment how antiquated Notes is and the methodology about them. This aspect of Accordance is behind the times.

 

Link to comment
Share on other sites

Something similar for me--Accordance kept deleting my User Notes (synced via Dropbox) on startup, for no discernible reason. Even trashing the startup session didn't help. So I have just been in a recent habit of recovering from Trash in Dropbox, but now my User Files are where they should be in Dropbox (via Finder), but won't open at all. I also have the .acc4 extension as well as .amynotes. Neither opens.

  • Like 1
Link to comment
Share on other sites

1 hour ago, Abram K-J said:

Something similar for me--Accordance kept deleting my User Notes (synced via Dropbox) on startup, for no discernible reason. Even trashing the startup session didn't help. So I have just been in a recent habit of recovering from Trash in Dropbox, but now my User Files are where they should be in Dropbox (via Finder), but won't open at all. I also have the .acc4 extension as well as .amynotes. Neither opens.

 

Yep. Exactly it. I had my files in my icloud documents folder. Was told this was the issue of syncing. So I followed the instructions of moving them. Then everything broke and sort of cascaded. Now I can't open any of them and the sync with iOS does even less.

 

It's so difficult. I hate being frustrated because I love Accordance. I wanted to start using my iPad more and realized there was this sync issue. Then it all broke, and apparently others have had similar issues. Losing years (or even if it was months) of info is an incredible problem.

Edited by jkdoyle
Link to comment
Share on other sites

I've always used Dropbox Sync. And now I'm trying the direct to mobile sync. I hangs up every time. 

 

Trying to work with my ipad (in it's botched up library), I noticed that the Christian Standard Bible wouldn't move. When I opened it, it would only the Ugaritic Data bank.  All jacked up.

So reinstalled Accordance completley on my ipad. Still does not sync. Hangs up on on mobile sync (on Accordance text modules, like HMT-W4). When I do a Dropbox sync, my notes don't sync correctly.

Link to comment
Share on other sites

Notes should be first entered into a trustworthy text editor or word processor (with backup files in separate locations), then copied to Accordance's Notes. It's more work upfront, but you shouldn't allow Accordance to hold your data hostage. It's buggy, and besides, I wouldn't trust any app to be around forever.

  • Like 2
Link to comment
Share on other sites

I've been experiencing similar issues and have dropped automatic sync with Dropbox, but I do use the manual sync. I've added extra security by suing Arq backup software for my Accordance folder. I can easily go back in time and retrieve files that are corrupted.

 

https://www.arqbackup.com/

  • Sad 1
Link to comment
Share on other sites

  • 3 weeks later...

I lost my User Notes today, for the second time 😒. In order to recover them the first time, I reinstalled Accordance in my Mac and Devices. It took me a lot of time to reorganize my Library. After a couple of days my notes showed up again, without missing anything. But today, after recording some lectures where I use my notes, I tried to sync with Dropbox and they got lost again. My file Bible Notes was erased from my software. I won't reinstall it again, and I see that others have had the same problem. Thus, my request is for Accordance Thecnical Support: please, inform us if that problem will be solved or not. Otherwise, I will have to star using another software for my notes, which I don't want to, of course. Thanks for letting me share this matter. 

  • Sad 1
Link to comment
Share on other sites

  • 2 weeks later...
On 4/18/2022 at 10:16 PM, William Phillips said:

Well I am sickened to write this but my user notes -- such as they were -- seem absolutely corrupted beyond any retrieval. Unfortunately the model used by concordance for these files (seemingly no proper way to export files, a confusing import and merge process that seems to just further corrupt what hope one had of retrieving files, a reliance on dropbox which is hopeless, and so forth) have resulted in utter confusion and apparently forever lost notes.

 

Although I suppose Technical Support could possibly still help I am appalled that such an important feature should be obscured into some opaque database that is coded in such a way as to make looking within impossible. And now I find when I even tried to merge user notes (in the vain hope that would accomplish something) it winds up deleting or perhaps just moving to the 'old' folder, the original -- but even those files display nothing.

 

Hopeless, confusing, infuriating. This methodology has now, in effect, stolen my own information!  I am very disappointed. I guess I may have to see  if any alternatives out there have a better, more understandable, standardized (as in file system manipulation) model. Even going to something like Logos (if any better) will not retrieve what I seem to have lost.

My notes and highlights have been (I thought) hopelessly corrupted several times. Missaligned, text transformed into something like wingdings, just totally trashed. Several times. I've worked with tech support on this on at least two occasions and they helped me recover much of what I thought was lost. I specifically remember working with Billy. But that was a few years ago; I don't know who's there now. I should also say that I now redundantly and religiously back up all my user files (beyond regular time machine backups). I suggest the same to everyone else if your work in accordance is worth the trouble for you. 

Link to comment
Share on other sites

Hi @A. Smith,

What you are describing about the Notes turning into Wingdings, typing over itself, etc is exactly how it is has been for me several times as well.

 

Yes, Billy is still here. He has always been really helpful and I appreciate working with him.

 

Regarding backups, I backup to TimeMachine at least once a day, but the problem is, the Highlights and Notes can get corrupted without you realizing it for awhile.

 

For example, lets say you are doing a bunch of Highlights and Notes for a while in Titus. You do your work, back up, do more work, backup. Everything seems to be going fine. However, as you are doing this Nehemiah is becoming corrupted beyond use. You can thus backup every day, every hour, it won't matter, since you are backing up the corruption.

 

In theory you can go to TimeMachine and find the last saved copy without corruption, but how can you do that? Import a backup, read the Bible cover to cover to check for corruption, and repeat until you find a non-corrupted backup?

 

Furthermore, I frankly can't find a non-corrupted copy. I went back two months and the problems are there also. I finally imported that, as this 2 month old copy is less corrupted, but obviously losing two months of work is another type of Note and Highlight corruption.

 

(On a side note, one thing I would like to mention is if you open your Notes as a parallel, and you are scrolling and as you do so the Notes start typing over themselves, don't panic. Just close the parallel and reopen it, and that will fix it 99% of the time. I don't know what is causing that, but it seems to be an unrelated, almost harmless issue.

 

Kristin

Link to comment
Share on other sites

11 hours ago, Kristin said:

it won't matter, since you are backing up the corruption.

That is a concern of mine too I have been pondering about. If you don't know that corruption is occurring automation continues backing up those corruptions and you only find out when trying to restore. So you need a know good last state backup in addition to automation. I usually have a completely separate backup copy, but have been thinking I need to be confident it is 100% clean and still thinking how to test and verify that.

 

Link to comment
Share on other sites

Hi @cweber,

That would be great if there was a way to test it. If you think of anything, let me know, but I think the only thing I can really do is hope the Accordance team is able to fix it soon.

 

Take care,

Kristin

Link to comment
Share on other sites

Most Accordance files are non-text files (Binary files of bytes representing something other than ASCII text data). it’s possible to read binary files using the right tools or using standard library functions such as with a programming language like Python or C#. However, it is not simple to do so.

 

There are few options for us customers:

  1. Try the Accordance Support SAFE mode recommendations
  2. If it is a library module that never changes and you have confidence it has not caused a crash, then you can keep a copy in backup that does not get updated and you can restore from that copy if necessary. This is where it would be nice if Accordance included a mechanism to verify our installed files against their repository, possibly using a simple checksum compare.
  3. For a file that varies in size, due to being updated with highlight preferences for example, you cannot look at the file size to compare with another version of the file (e.g. a backup). I can only think to keep disparate copies in backup from a "last known" non-crash causing experience and restore from that if you experience a crash. Not easy to do or know that last state and you can end up many versions behind and lose a lot of preferences or edits.
  4. Keep a "copy" of all Notes in another application editor you can restore (copy and paste) from.

As a comparison, technically harvesting metadata from the XML type Tool and Text files was not overly difficult and proved useful to me for sorting out my purchased libraries. However, something as desirable as obtaining Authorship proved enough of a nuisance I haven't revisited that. In summary, solving binary file-based crash issues is rather difficult without developer insight and access. I do not think this situation will change without a strategic change in direction for the application. Hopefully the Accordance team listen and improve this serious limitation for us all.

 

Edited by cweber
Link to comment
Share on other sites

Hi @cweber,

Thank you for the explanation. That is all very interesting. If I understand correctly, it sounds like you think that texts (such as the ESV or whatever) is built on one code which is not having issues, while things like Notes and Highlights are built on a completely different technology which can only handle so much. Thus, the corruption issues with Notes and Highlights is not due to a bug, per se, but rather because the specific technology Notes and Highlights use are fundamentally not as stable. So, the only way to fix the problem with Notes and Highlights would be to totally change the technology they use which is a major commotion.

 

I know nothing about creating programs, but from what you are saying, it sounds like Highlight and Note file corruption is "expected" because there is not anything the tech team or developers can do to prevent it, given limits of the specific technology used. Is this correct?

 

If that is correct, do you think it is just totally random, and little note files can easily become corrupted? Or do you think small note files are fairly safe? I keep A LOT of important data on notes and really need them to be secure, as I think a lot of people on this forum would also say.

 

 

Link to comment
Share on other sites

@Kristin I have possibly confused you as I have a tendency to speak to technically :)

 

The main difference between modules (Text and Tools) are that those files once downloaded, do not change (at least I do not think they do) and these type of files remain read-only. Because Texts and Tools are read-only, once you download and install a good one of those files I would not expect an issue due to reading from them.

 

However, our personal highlights and notes are both read-write types of files, they are written to and read from. When writing to files, if they are in use or open at the time the software can incur a situation (similar to what occurs in databases) called locking that prevents writing to the personal notes or highlight file. Not until the lock is lifted can Accordance then write your changes. It is usually during the write process corruption may occur.

 

Therefore, the only time I anticipate corrupted Text/Tools is that occurring during download. However, depending upon the software implementation for personal preference (highlighting/notes) write and read of these files, combined with operating system activities, I would not be surprised to see those files incur corruption occasionally. These are the files I believe we need to have last known good state backups for and separate from automated backups.

 

 

  • Like 1
Link to comment
Share on other sites

PS: The latest Accordance 14 announced feature "Record your answers to workbook and discussion questions right in the book itself" has me concerned the Text/Tools may also be write and read. If so, those can be subjected to corruption also. I do not know what books are referred to here :)

Edited by cweber
Link to comment
Share on other sites

Hi @cweber,

Concerning your previous response, thank you for the explanation. That was helpful, and makes perfect sense. :)

 

Regarding what you just wrote now, I like Accordance enough that I went ahead and have already pre-ordered v.14 (which I think was the right decision), but I have actually been pretty fuzzy on all of the changes. Do you happen to have a link showing the changes in 14?

 

Given what you just wrote makes me REALLY concerned. If the corruptions to the Notes and Highlights is regarding the read/write function (and this is an issue which has literally cost me months more than once), it is a really concerning thought that that wide-spread corruption could start damaging actual biblical or other ancient texts. I feel like before officially downloading v.14 I would need clarification if it will make the texts read/write also, since the corruption is still so serious with the Notes and Highlights. I am not sure who to ask though, as it is not really a "tech support" question, per se.

 

Kristin

Link to comment
Share on other sites

@R. Mansfield  could you please chime into this conversation and offer some clarity surrounding what files are subject to possible corruption?

 

What we need to understand is the risk, what files do we need to be mindful of and therefore backup suitably. We only ask that you consult with your developers and provide a brief for what is subject to corruption and thereby what we need to do to backup and recover from, in the event of such a situation. For us customers (with some substantial investment) this is a VERY significant concern and real negative experience, yet we are speculating instead of being guided by Accordance on such an important issue and concern.

 

 

 

Edited by cweber
  • Like 3
Link to comment
Share on other sites

On 4/18/2022 at 2:16 PM, William Phillips said:

I am appalled that such an important feature should be obscured into some opaque database that is coded in such a way as to make looking within impossible.

 

I agree with this wholeheartedly.  This dates back to decisions made long before I started as a programmer here, and maybe they were the best choices when my predecessors made them.  But I do want to switch user notes to something simple and human readable and hard to corrupt.  Markdown is one possibility I've considered.

 

Given the concerns voiced in this thread, I'll go ahead and attach the documentation we have about the byte level structure of an .amynote file.  Maybe it will be useful to someone here in the future.

 

If anyone can post an example of one of the corrupted notes files, or give me insight into the exact sequence of events that causes corruption, it would go a long way towards fixing this.

byte_layout_of_amynote_files.pdf

  • Like 5
  • Thanks 1
Link to comment
Share on other sites

16 hours ago, cweber said:

@R. Mansfield  could you please chime into this conversation...

 

What Silas said☝️

  • Like 1
Link to comment
Share on other sites

46 minutes ago, Silas Marrs said:

If anyone can post an example of one of the corrupted notes files, or give me insight into the exact sequence of events that causes corruption, it would go a long way towards fixing this.

 

Hi @Silas Marrs,

Thank you for the explanation, and I am grateful to hear that resolving this is a priority. Regarding the examples, I have personally sent corrupted Notes and Highlights to tech support on a few occasions, so I probably could, but I am not 100% sure if I understand what you need.

 

Would it be helpful if I....

 

1) find the old emails with the corrupted files attached?

2) provide screenshots?

 

3) give more details concerning the corrupted files I have already sent (such as, "the word 'peace' in such and such a verse has a highlight I never set"

4) what you wrote is actually more addressing technical specs and addressed to people on the forum who understand that document you attached.

Let me know if I can help in any way.

 

Kristin

Link to comment
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
×
×
  • Create New...