script based FileMigrationLoader

[chb0github]

The MigrationLoader and the FileMigrationLoaderFactory make things really flexible. But, I would like to propose another implementation that will really open up mybatis to non-java developers and allow for a light-touch

Some background:

The situation I am trying to address requires a custom MigrationLoader. And, it seems to be working the way I would like it to be. But, let's consider what I will need to do to take advantage of it

1. Create a java project: (pom.xml, directory structures, etc.) because I have to pull in the mybatis interfaces
2. Build a jar
3. Release it to an artifact repository
4. Deploy it. Which, in the case of migrations, means I would have to put it in the drivers directory since I know this will get picked up on the class path. Should I create a custom migrations script to check to see if it has the latest migration loader I wrote before running and download it if needed (including getting around any security issues I might encounter while checking with the artifact repo)? Or, should I just check in the jar into the migration project under drivers and update as needed?

The question thus arises: Where should I put this Java project that consists entirely of 2 java files and 1 meta file?

Should I:

1. Create a new repo internally to manage it (and a build and release process)
2. Create maven/java directory structures in the current repo containing nothing but sql scripts meant for migrations? (Which is only really going to be used by DBAs).

Option 1 is really a heavy weight process to get about 40 lines of code in use
Option 2 is bound to really confuse people who are totally unfamiliar with java.

It would really open up migrations to non-java developers (or java developers where a heavy touch process is not desired). We could use the existing scripting configuration/functionality.

I am willing to make the changes and make sure they are backward compatible. I would just like to donate it to the community because I think it would allow for a very light foot-print when customizing and because it would be a stand-out feature from all the other migration tools.

[harawata]

Thank you for the idea, @chb0github !
I thought about it, but it 1) requires another section in the documentation and 2) complicates the implementation more or less, so I think the current implementation is good enough for a first release.
Let's keep this ticket open and see if it gets other users interests before you start working on it.

To all,
Please use the 'reaction' feature to vote up instead of a '+1' comment.

[chb0github]

@harawata

1. Documentation: If you can point me to the docs that need updating I will gladly do it
2. This I don't understand at all. It's an entirely new and independent implementation. How could it complicate anything if it's independent?

Your reply also didn't address any of my points:

1. Heavy weight build process.
2. Supporting scripts opens it to non-java devs!

[h3adache]

What do you achieve from having this in migrations core vs just adding it as your own ScriptBasedMigrationLoader other than having to be supported in migration core? I can think of a few migration loaders that might be needed (File which is currently there, Directory based one which has been requested twice that I counted, a Jar based on that I use in my own personal development and the java based one). A script based one is just another of those vs a consolidation and simplification of the process of migration loading.

I'm never in favor of complicating especially when it's just to support one off style processes.

[chb0github]

I am not working with a Java-based migration project. If I have to implement a FileMigrationLoader then I need to take my non-java project and java-ify it. See above for details.

This approach actually would allow migrations to essentially end the one-off requests. Migrations greatest features are it's commendable flexibility; it doesn't get any better than this.

I think the part that seems to be missing is that: Not all projects using migrations are java based and opening it up to not requiring Java dev skills is really unique and gives migrations an edge against the competition. Frankly, it's a killer feature IMO that can be implemented fairly straightforwardly.

I am a seasoned vet - I'm not proposing some hack and of course, code review it. If it's a matter of support, then I will continue to support it even after release. Though again, it's going to be a pretty easy implementation so I don't imagine there should be any issues.

I am trying to be a positive contributor and I think this is a feature that will generate many stars. Thanks for considering.

[hibaymj]

@h3adache I am having trouble seeing where this is 'one-off', when you acknowledge there are not only a few people asking for similar features, but offering to have their implementations form the foundation of the features themselves.

This feature addresses a legitimate pain with the use of mybatis, namely it needs to be included within a Java based project. The functionality of the tool is excellent, and the applicability of the script based non-annotated invocation of the functionality could be very wide within the software development field.

The benefit to describing rejection of this feature as a simplification is dubious, as it is implemented with interfaces in order to allow this type of extension to be modularized, and independent.

@harawata If @chb0github would like to contribute the documentation fixes as well, this should leave your 2) as your only concern. In which case I would refer to my point above, how does this complicate the implementation at all? The only concern I can see is if this parser could in some way introduce a security vulnerability through the scripting interface.

[litwicki]

This would essentially open up mybatis to the entire LAMP stack universe; I'm failing to see the downside..

[chb0github]

php
ruby
lua

[h3adache]

@hibaymj @litwicki I appreciate the enthusiasm. But I fail to see how it's currently NOT opened up to those other stacks. I use it myself in many non java/jvm projects (js, perl, python, c++).
With the addition of the hooks that @harawata added, there's already scripting language support for all the heavy lifting that migrations does.

@hibaymj again, appreciate the feedback but I said directory based loading was what was requested multiple times and that is now handled by @harawata's commit and can be a drop in plugin to migrations. Please read what was written before attacking me.

[chb0github]

@h3adache - in a C++ project: Do you have a custom FileMigrationLoader implemented in C++? It's not that it can't be used in those projects, it just can't be used with a customization. To use a custom FileMigrationLoader in your C++ project you would have to build it in java and put the resulting jar somewhere on the CP

Last I heard @harawata hadn't merged in those changes yet. And, that's handy. He actually opened it up to support his use case. Kudos.

[chb0github]

@h3adache Maybe @hibaymj was a bit harsh. I think the point is: FileMigrationLoader and MigrationLoader are interfaces. Which means they are designed to be independent and thus low-maintenance

[hibaymj]

@h3adache Apologies for the apparently heavy handed response, no attack was initiated or intended. I had to spend the previous 5 hours doing HR related training, nerves were a bit raw apparently.

What I believe @chb0github is saying is that migrations assumes there is a Java build tool being used to build the project, which is clearly not the case when using NPM, Yarn, Phing, Deployer, etc... You would still have the dependency of a JVM in the environment to execute migrations, however you wouldn't be required to use a Java based build tool for your project to build the custom project loader. This is the limitation I believe this issue is attempting to address.

Do the hooks you referenced by @harawata address this concern and allow this flexibility?

[harawata]

@chb0github
I have to assume that you asked your friends (who obviously do not understand the purpose of this request correctly) to vote this issue up to force us to accept your contribution.
It makes it difficult for us to evaluate what really is beneficial to this project and also is disrespectful to users who actually use this tool.
More than anything, it makes me really disappointed to see a respected developer like you do such a thing.

We try to be deliberate in choosing what to add (e.g. Does it benefit many users? Is it within the scope of the project?, etc.) because, while adding a new feature/option is relatively easy and can be done at any time, it’s quite difficult to remove it once it’s published and it has to be maintained.
If you are not familiar with this idea, here are a few good reads.
https://lord.io/blog/2014/oss-tips/
https://opensource.guide/best-practices/#learning-to-say-no

It is understandable that you feel frustrated to see your idea/PR/request is rejected and it may be our fault if our explanation is not satisfactory.
But we committers are trying to do what is best for the project and if you cannot believe it, you would have to create a new project of your own.

[chb0github]

@hibaymj turned me on to your project a couple of months ago and I asked him what he thought of this idea. His words are his own and I didn't put him up to this at all. I am working with one other person and he is watching the several issues I have open as it effects him to. The fact remains that I am not working with a Java project and the other people who will work with the project when I am done know nothing about compiling or deploying Java. To use a custom `FileMigrationLoader` would require a Java project and build and the subsequent jar placed in the project classpath. Infact, the entire company I work at is mostly unfamiliar with Java. I recommended this feature because I believe it really opens up possibilities with distinctive functionality that no other migration tool has. When I was evaluating tools for db migrations any that were code only approaches I had to reject because the users of the project are dbas. 1. So it had to have strong SQL support. 2. It also had to have rollback functionality 3. It needed to have pre and post validate and this is where hooks prove handy 4. For reasons too baffling even for me to explain, we have 1 app sitting on anywhere between 1 and 3 databases depending on where it's deployed; with schema and tables split arbitrarily between them. WTF?! You say? This is where the custom migration loaders save the day. However, they currently require being written in Java; see above. I am sorry the commentary has gotten out of hand and you should make the decision best for your user base. I do politely ask that you continue to evaluate it based on merits. Nothing but respect for all of your efforts and keep up the good work.

…

On Tue, Jan 2, 2018, 10:24 PM Iwao AVE! ***@***.***> wrote: @chb0github <https://github.com/chb0github> I have to assume that you asked your friends (who obviously do not understand the purpose of this request correctly) to vote this issue up to force us to accept your contribution. It makes it difficult for us to evaluate what really is beneficial to this project and also is disrespectful to users who actually use this tool. More than anything, it makes me really disappointed to see a respected developer like you do such a thing. We try to be deliberate in choosing what to add (e.g. Does it benefit to many users? Is it within the scope of the project?, etc.) because, while adding a new feature/option is relatively easy and can be done at any time, it’s quite difficult to remove it once it’s published and it has to be maintained. If you are not familiar with this idea, here are a few good reads. https://lord.io/blog/2014/oss-tips/ https://opensource.guide/best-practices/#learning-to-say-no It is understandable that you feel frustrated to see your idea/PR/request is rejected and it may be our fault if our explanation is not satisfactory. But we committers are trying to do what is best for the project and if you cannot believe it, you would have to create a new project of your own. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#112 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABaI0KtFZLZuE7BaeLz_mhaW1EsOMRIVks5tGx0LgaJpZM4RP99o> .

[hazendaz]

Count me as uninformed. This is for my learning benefit. How exactly do you even end up using something written in java even if just an interface without actually using the jvm? It's still byte code at that point. I also fail to see how other tools could not be used with underlying users needing to learn java. I don't think they need too. There is a maven wrapper. You can plug in frontend maven plugin and it will handle many things like yarn. So at best a maven execution is all someone might need without knowing what that does. Really for my own benefit as a dev/ops guy trying to understand that piece and how it would work. A sample demo would be nice to look at. My shop is 50/50 java and . Net so this understanding might be very useful to me. I'm also going to see if I can get a dba guru I work with to look at all this. I think to everyone else's broader point it's a lot of extra code we end up having to learn support and otherwise. If you could build on your own project as many do that with mybatis then release to central you will start getting monthly sonatype statistics. A link from us is much easier to support initially and let's say for argument sake that 1000 people with individual ip addresses download you project a month and it grows. That would turn the tides I believe. But on the other hand what if it's only you using it, it gets no forks, no stars, etc. And there I think lies the concern. Keep in mind I'm really dev/ops in this regard so I cannot make a true informed decision on the existing code base let alone new logic. I just know it's big and that always raised concerns. And it's certainly a balancing act as no one ever wants to scare away contributors. I work many projects were everyone else left and it's really tough. Mybatis has a huge group of great people and I find it extremely rare that is the case so I trust each and everyone on the team and what they have to say either way. I know many on the team have side mybatis things that are not with the team and sometimes a few years go by before those land with the team. I think that goes to the statistics gathering where there might otherwise be concern. Get Outlook for Android<https://aka.ms/ghei36>

________________________________ From: Christian Bongiorno <notifications@github.com> Sent: Wednesday, January 3, 2018 2:31:15 AM To: mybatis/migrations Cc: Subscribed Subject: Re: [mybatis/migrations] script based FileMigrationLoader (#112) @hibaymj turned me on to your project a couple of months ago and I asked him what he thought of this idea. His words are his own and I didn't put him up to this at all. I am working with one other person and he is watching the several issues I have open as it effects him to. The fact remains that I am not working with a Java project and the other people who will work with the project when I am done know nothing about compiling or deploying Java. To use a custom `FileMigrationLoader` would require a Java project and build and the subsequent jar placed in the project classpath. Infact, the entire company I work at is mostly unfamiliar with Java. I recommended this feature because I believe it really opens up possibilities with distinctive functionality that no other migration tool has. When I was evaluating tools for db migrations any that were code only approaches I had to reject because the users of the project are dbas. 1. So it had to have strong SQL support. 2. It also had to have rollback functionality 3. It needed to have pre and post validate and this is where hooks prove handy 4. For reasons too baffling even for me to explain, we have 1 app sitting on anywhere between 1 and 3 databases depending on where it's deployed; with schema and tables split arbitrarily between them. WTF?! You say? This is where the custom migration loaders save the day. However, they currently require being written in Java; see above. I am sorry the commentary has gotten out of hand and you should make the decision best for your user base. I do politely ask that you continue to evaluate it based on merits. Nothing but respect for all of your efforts and keep up the good work.

On Tue, Jan 2, 2018, 10:24 PM Iwao AVE! ***@***.***> wrote: @chb0github <https://github.com/chb0github> I have to assume that you asked your friends (who obviously do not understand the purpose of this request correctly) to vote this issue up to force us to accept your contribution. It makes it difficult for us to evaluate what really is beneficial to this project and also is disrespectful to users who actually use this tool. More than anything, it makes me really disappointed to see a respected developer like you do such a thing. We try to be deliberate in choosing what to add (e.g. Does it benefit to many users? Is it within the scope of the project?, etc.) because, while adding a new feature/option is relatively easy and can be done at any time, it’s quite difficult to remove it once it’s published and it has to be maintained. If you are not familiar with this idea, here are a few good reads. https://lord.io/blog/2014/oss-tips/ https://opensource.guide/best-practices/#learning-to-say-no It is understandable that you feel frustrated to see your idea/PR/request is rejected and it may be our fault if our explanation is not satisfactory. But we committers are trying to do what is best for the project and if you cannot believe it, you would have to create a new project of your own. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#112 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABaI0KtFZLZuE7BaeLz_mhaW1EsOMRIVks5tGx0LgaJpZM4RP99o> .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#112 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AA7ho3swsIoXEkHNajOB-3LB9EXRp4Tzks5tGyzDgaJpZM4RP99o>.